Assessing an assessment tool: Analysis of multiple choice questions on difficulty level and discrimination power, from an assessment in physiology

Background: Multiple Choice Questions (MCQs) or ‘‘items” are frequently used in medical education to assess students’ knowledge at the end of an academic session. However, item analysis has to be done to confirm their excellence. This piece of work was aimed to analyze test items in the subject of Physiology. In view of the fact that item analysis has never been conducted in our physiology department, thus this study was planned to assess the quality of MCQs and construct a viable question bank. Aim and Objectives: To evaluate the quality of MCQs used so as to develop a pool of valid items to update the question bank. Materials and Methods: Total of one hundred 1st year MBBS students were given 40 MCQs. Analysis was done with the item difficulty and item discrimination indices. Results: Difficulty indices (p) of 14 items were easy while about two MCQ were difficult and the remaining 24 of the items were within acceptable range. The discrimination index (DI) (D) for 40 items; one item showed negative discrimination, ten were having poor discrimination, whereas 11 were having excellent discrimination.Nine items were having acceptable, and nine items were having good discrimination. The DI exhibited slight positive correlation with difficulty index (r = 0.1575 P = 0.05). However, it was not statistically significant. Conclusion: The findings show that most of the items were falling in acceptable range of difficulty and discrimination level. It concludes that Item analysis is a very useful and necessary procedure to obtain valid Items from time to time.

Nivel de dificultad y poder de discriminación del examen final de la asignatura Ontogenia Humana y SOMA / Difficulty level and discernment power on Human Ontogeny and SOMA final test subject

RESUMEN Fundamento: La calidad de los instrumentos evaluativos resulta esencial en el proceso de evaluación del aprendizaje. Objetivo: Determinar los índices de dificultad y discriminación del examen final ordinario de la asignatura Ontogenia Humana y Soma. Metodología: Se realizó un estudio retrospectivo transversal en la asignatura Ontogenia y SOMA, del curso 2017-2018 en la Universidad de Ciencias Médicas de Sancti Spíritus, en el que se procesaron 163 exámenes ordinarios finales, el 30 % de los examinados seleccionados mediante un muestreo aleatorio estratificado por grupos, se calcularon el índice de dificultad e índice de discriminación por preguntas, temas y ciencias. Resultados: El número de incisos esperados y reales, de acuerdo con las horas clases para cada ciencia estuvo ajustado, no así en las temáticas Ontogenia de 17 incisos esperados se dedicaron 12 (70.5 %), en SOMA de 34 incisos esperados se dedicaron 29 (85.2 %); el índice de dificultad por temáticas, ciencias y temarios fue medianamente fácil (0.74-0.86) al igual que para el examen en su conjunto. El índice de discriminación del tema Ontogenia fue superior en el temario 1 (T1: 0.37 vs. T2: 0.24) y similar en SOMA (T1: 0.40 vs. T2: 0.39) y a nivel de las ciencias [Embriología (T1: 0.39 vs. T2: 0.31), Anatomía (T1: 0.39 vs. T2: 0.37)]. A nivel de preguntas, el índice de discriminación más bajo lo tuvo la pregunta 1 del primer temario (0.22), el resto tuvo índices de discriminación superiores a 0.30. Conclusiones: Ambos temarios tienen un índice de discriminación similar. El índice de dificultad fue medianamente fácil.

ABSTRACT Background: The quality of the assessment instruments is essential in the learning assessment process. Objective: To determine the difficulty and discernment indexes of the Human Ontogeny and SOMA ordinary final test subject. Methodology: A cross-sectional retrospective study was conducted on the Ontogeny and SOMA subject, from 2017 to 2018 academic year at the Sancti Spíritus Faculty of Medical Sciences, 163 final ordinary tests were processed, 30 % of the examinees selected by stratified random cluster sampling, the difficulty and discernment index were calculated by questions, topics and sciences. Results: The number of expected and real items, according to the class hours for each science, was adjusted, but not in the subjects (Ontogeny of 17 expected items were dedicated 12 (70.5 %), in SOMA of 34 expected items 29 were dedicated (85.2 %), the difficulty index by subjects, sciences and syllabus was moderately easy (0.74-0.86) as for the exam as a whole. The discernment index of the topic Ontogeny was higher in the syllabus 1 (T1: 0.37 vs. T2: 0.24) and similar in SOMA (T1: 0.40 vs. T2: 0.39) and [Embryology (T1: 0.39 vs. T2: 0.31), Anatomy (T1: 0.39 vs. T2: 0.37)]. At the science level questions, the lowest discernment index was found in question 1 from the first syllabus (0.22), the rest had discernment indexes higher than 0.30. Conclusions: Both topics have a similar discernment index. The difficulty index was moderately easy.

Role of Multiple Choice Questions in Assessment of Competency of Knowledge in Anatomy

Objectives: The main objective of the current study is to examine the items using the item analysis to assess their competency of knowledge in human anatomy. Methods: This is a quantitative descriptive cross-sectional study conducted at the Najran University faculty of Medicine, in the department of anatomy. A 31-second year MBBS students of level three took the multiple-choice question exam comprising 100 questions in one session. Each item was analyzed by item analysis. The planned period of data collection is the first semester of the academic year 2019-2020. The data were analyzed using SPSS version 25, and Excel Software. All analyses were performed using the descriptive frequency, explore and correlation probabilities a p-value, p<0.01 (2-sided) was considered to indicate statistical significance. Results: The assessment of test items use the item analysis that is very difficult question: 25; 25%, difficult question: 8; 8%, average question: 46; 46%, easy questions; 9; 9% and very easy: 9; 9%. The discrimination indices, which are poor discrimination; 34; 34%, acceptable discrimination; 11; 11%,good discrimination; 6; 6%, very good discrimination; 4; 4% and excellent discrimination; 45; 45%. The performance of students in this test showing that a large number of items have a role in distinguishing students knowledge in the anatomy test, but in Simple parts of the scale that the items did not succeed in assessing students. Items of anatomy exams displayed a strong relationship of difficulty (P) and discrimination (D) indices. Conclusion: The MCQs items assess their competency of knowledge in human anatomy. The competence of students in the knowledge of anatomy measured at a high rate of A-type context-rich MCQs can be stem is case scenario, lead-in; question and options; distractors, and key. These items can differentiate good and poor accomplishment students..

Índice de dificultad y discriminación de ítems para la evaluación en asignaturas básicas de medicina / Difficulty and discrimination index of items for evaluation in basic medical subjects

Introducción: La determinación de la calidad de un instrumento de medición implica la evaluación tanto de la exactitud como de la estabilidad de dicha medición o el resultado. Objetivo: Evaluar el índice de dificultad y discriminación de los ítems de los instrumentos de medición de algunas asignaturas del eje curricular de Fundamentos de la medicina de una escuela de medicina particular. Métodos: Mediante estudio observacional, comparativo, transversal y prolectivo, se analizaron los ítems del examen ordinario de las asignaturas Morfología, Bioquímica y Biología Molecular, y Microbiología y Virología Médica, los cuales fueron contestados por 112 alumnos de Morfología, 101 de Bioquímica y Biología Molecular y 89 de Microbiología y Virología Médica. El número de reactivos resultó 100 para Morfología, 80 para Bioquímica y Biología Molecular y 75 para Microbiología y Virología Médica. Se utilizó estadística descriptiva, y se calculó el índice de dificultad y el índice de discriminación de los ítems y de todo el examen. Además, se utilizó ANOVA, considerando estadísticamente significativo un resultado igual o menor a 0,05. Resultados: Los ítems resultaron más difíciles en el examen de Morfología, ya que el 19 por ciento correspondió a la clasificación de relativamente difíciles y el 50 por ciento a la de difíciles, con un total del 69 por ciento entre relativamente difíciles y difíciles. El índice de discriminación resultó mayor en la asignatura de Microbiología y Virología Médica, mientras que en Bioquímica y Biología Molecular se presentó una buena capacidad de discriminación. Conclusiones: Existió variabilidad en el grado de dificultad y el poder de discriminación de los ítems de los exámenes analizados(AU)

Introduction: Determining the quality of a measuring instrument implies the evaluation of both the accuracy and the stability of such measurement or result. Objective: To evaluate the index of difficulty and discrimination of the items of the measuring instruments of some subjects of the curricular axis of Fundamentals of Medicine of a particular medical school. Methods: By means of an observational, comparative, cross-sectional and prolective study, the items of the ordinary examination of the subjects Morphology, Biochemistry and Molecular Biology, and Microbiology and Medical Virology were analyzed, which were answered by 112 students of Morphology, 101 of Biochemistry and Biology Molecular, and 89 of Microbiology and Medical Virology. The number of reagents was 100 for Morphology, 80 for Biochemistry and Molecular Biology, and 75 for Microbiology and Medical Virology. Descriptive statistics were used, and the difficulty index and the discrimination index of the items and of the entire exam were calculated. In addition, ANOVA was used, considering a statistically significant result equal to or less than 0.05. Results: The items were more difficult in the Morphology test, since 19 percent corresponded to the classification of relatively difficulty and 50 percent corresponded to those of difficulty, with a total of 69 percent between relative difficulty and difficulty. The discrimination index was higher in the subject of Microbiology and Medical Virology, while in Biochemistry and Molecular Biology there was a good capacity for discrimination. Conclusions: There was variability in the degree of difficulty and the power of discrimination of the items of the analyzed exams(AU)

Item analysis as tool to validate multiple choice question bank in pharmacology

Background: Multiple choice questions (MCQs) are a common method for formative and summative assessment of medical students. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), distracter efficiency (DE). The objective of this study was to assess the quality of MCQs currently in use in pharmacology by item analysis and develop a MCQ bank with quality items.Methods: This cross-sectional study was conducted in 148 second year MBBS students at NKP Salve institute of medical sciences from January 2018 to August 2018. Forty MCQs twenty each from the two term examination of pharmacology were taken for item analysis A correct response to an item was awarded one mark and each incorrect response was awarded zero. Each item was analyzed using Microsoft excel sheet for three parameters such as DIF I, DI, and DE.Results: In present study mean and standard deviation (SD) for Difficulty index (%) Discrimination index (%) and Distractor efficiency (%) were 64.54±19.63, 0.26±0.16 and 66.54±34.59 respectively. Out of 40 items large number of MCQs has acceptable level of DIF (70%) and good in discriminating higher and lower ability students DI (77.5%). Distractor efficiency related to presence of zero or 1 non-functional distrator (NFD) is 80%.Conclusions: The study showed that item analysis is a valid tool to identify quality items which regularly incorporated can help to develop a very useful, valid and a reliable question bank.

Analysis of multiple choice questions from a formative assessment of medical students of a medical college in Delhi, India

Background: Assessment is a dominant motivator to direct and drive students learning. Different methods of assessment are used to assess medical knowledge in undergraduate medical education. Multiple choice questions (MCQs) are being used increasingly due to their higher reliability, validity, and ease of scoring. Item analysis enables identifying good MCQs based on difficulty index (DIF I), discrimination index (DI), and distracter efficiency (DE).Methods: Students of second year MBBS appeared in a formative assessment test, that was comprised of 50 “One best response type” MCQs of 50 marks without negative marking. All MCQs were having single stem with four options including, one being correct answer and other three incorrect alternatives (distracter). Three question paper sets were prepared by disorganizing sequence of questions. One of the three paper sets was given to each student to avoid copying from neighboring students. Total 50 MCQs and 150 distracters were analyzed and indices like DIF I, DI, and DE were calculated.Results: Total Score of 87 students ranged from 17 to 48 (out of total 50). Mean for difficulty index (DIF I) (%) was 71.6+19.4. 28% MCQs were average and “recommended” (DIF I 30-70%). Mean for discrimination index (DI) was 0.3+0.17. 16% MCQs were “good” and 50% MCQs were in “excellent” criteria, while rests of the MCQs were “discard/poor” according to DI criteria. Mean for distracter efficiency (DE) (%) was 63.4+33.3. 90% of the items were having DE from 100 to 33%. It was found that MCQs with lower difficulty index (<70) were having higher distracter efficiency (93.8% vs. 6.2%, p=0.004).Conclusions: Item analysis provided necessary data for improvement in question formulation and helped in revising and improving the quality of items and test also. Questions having lower difficulty index (<70) were significantly associated with higher discrimination index (>0.15) and higher distractor efficiency.

The relationship between classical item characteristics and item response time on computer-based testing / 한국의학교육

PURPOSE: This study investigated the relationship between the item response time (iRT) and classic item analysis indicators obtained from computer-based test (CBT) results and deduce students' problem-solving behavior using the relationship. METHODS: We retrospectively analyzed the results of the Comprehensive Basic Medical Sciences Examination conducted for 5 years by a CBT system in Dankook University College of Medicine. iRT is defined as the time spent to answer the question. The discrimination index and the difficulty level were used to analyze the items using classical test theory (CTT). The relationship of iRT and the CTT were investigated using a correlation analysis. An analysis of variance was performed to identify the difference between iRT and difficulty level. A regression analysis was conducted to examine the effect of the difficulty index and discrimination index on iRT. RESULTS: iRT increases with increasing difficulty index, and iRT tends to decrease with increasing discrimination index. The students' effort is increased when they solve difficult items but reduced when they are confronted with items with a high discrimination. The students' test effort represented by iRT was properly maintained when the items have a 'desirable' difficulty and a 'good' discrimination. CONCLUSION: The results of our study show that an adequate degree of item difficulty and discrimination is required to increase students' motivation. It might be inferred that with the combination of CTT and iRT, we can gain insights about the quality of the examination and test behaviors of the students, which can provide us with more powerful tools to improve them.

Spontaneous bone regeneration after surgical extraction of a horizontally impacted mandibular third molar: a retrospective panoramic radiograph analysis / 대한악안면성형재건외과학회지

BACKGROUND: The mandibular third molar (M3) is typically the last permanent tooth to erupt because of insufficient space and thick soft tissues covering its surface. Problems such as alveolar bone loss, development of a periodontal pocket, exposure of cementum, gingival recession, and dental caries can be found in the adjacent second molars (M2) following M3 extraction. The specific aims of the study were to assess the amount and rate of bone regeneration on the distal surface of M2 and to evaluate the aspects of bone regeneration in terms of varying degree of impaction. METHODS: Four series of panoramic radiographic images were obtained from the selected cases, including images from the first visit, immediately after extraction, 6 weeks, and 6 months after extraction. ImageJ software® (NIH, USA) was used to measure linear distance from the region of interest to the distal root of the adjacent M2. Radiographic infrabony defect (RID) values were calculated from the measured radiographic bone height and cementoenamel junction with distortion compensation. Repeated measures of analysis of variance and one-way analysis of variance were conducted to analyze the statistical significant difference between RID and time, and a Spearman correlation test was conducted to assess the relationship between Pederson’s difficulty index (DI) and RID. RESULTS: A large RID (> 6 mm) can be reduced gradually and consistently over time. More than half of the samples recovered nearly to their normal healthy condition (RID ≤ 3 mm) by the 6-month follow-up. DI affected the first 6 weeks of post-extraction period and only showed a significant positive correlation with respect to the difference between baseline and final RID. CONCLUSIONS: Additional treatments on M2 for a minimum of 6 months after an M3 extraction could be recommended. Although DI may affect bone regeneration during the early healing period, further study is required to elucidate any possible factors associated with the healing process. The DI does not cause any long-term adverse effects on bone regeneration after surgical extraction.

Dificultad en evaluaciones frecuentes y finales en contenidos de Anatomía de Ontogenia Humana y SOMA / Difficulty of Quizzes and Final Evaluations in Contents about Anatomy of Human Ontogeny and Osteo-Muscular-Articulatory System

Introducción: En el marco de la validación del plan de estudio D de la carrera de Medicina en Cuba, se analiza la calidad de los instrumentos evaluativos aplicados en asignaturas del nuevo plan de estudio. Objetivo: Comparar el nivel de dificultad de las evaluaciones frecuentes y finales de la asignatura Ontogenia Humana y SOMA, en el contenido correspondiente a la anatomía de los músculos. Métodos: Se realizó una investigación pedagógica en la Universidad de Ciencias Médicas de Matanzas durante el curso 2016-2017. Se analizaron 2763 preguntas escritas de clases prácticas y 553 exámenes ordinarios. Se aplicaron métodos teóricos, empíricos y procesamientos estadísticos de datos. Resultados: Predominó el nivel medianamente fácil en las preguntas de las evaluaciones frecuentes, con una media en 0,78. El resultado estuvo influenciado por el diseño de complejidad ascendente en la forma de abordar los contenidos en dichas evaluaciones, que favoreció un entrenamiento en los estudiantes sobre la forma de evaluar. En el examen final el 65 por ceinto de los incisos correspondientes a la pregunta de anatomía de los músculos presentó dificultad media. De dificultad medianamente fácil resultó el 20 por ciento de los incisos y el 15 por ciento restante medianamente difícil. Conclusiones: El nivel de dificultad del examen final fue mayor que en las evaluaciones parciales en el 80 por ciento de los temas y el cambio fue de una categoría a la siguiente. La evaluación del contenido de músculos en la asignatura se correspondió con un nivel adecuado al cumplir con los requisitos de calidad que exige la acreditación(AU)

Introduction: During the validation process for the study plan D of the medical major in Cuba, the quality is analyzed of the evaluative tools applied in subjects of the new study plan. Objective: To compare the level of difficulty of quizzes and final evaluations of the subject Human Ontogeny and Osteo-Muscular-Articulatory System, regarding the content for the anatomy of the muscles. Methods: A pedagogical research was carried out at the University of Medical Sciences of Matanzas during the academic year 2016-2017. We analyzed 2763 quizzes from workshops and 553 first-call exams. Theoretical and empirical methods, together with statistical data processing were applied. Results: The moderately easy level prevailed in the questions of quizzes, with a mean of 0.78. The result was influenced by the design of ascending complexity in the way of addressing the contents in such evaluations, which favored a training in the students regarding the way to evaluate. In the final exam, 65 percent of the items corresponding to the question of anatomy of the muscles presented average difficulty, while 20 percent of the items were of moderately easy difficulty and the other 15 percent were moderately difficult. Conclusions: The level of difficulty of the final exam was higher than in quizzes in 80 percent of the subjects and the change was from one category to the next. The evaluation of the content of muscles in the subject corresponded to an adequate level, since it met the quality requirements for accreditation(AU)

Item analysis of multiple choice questions of undergraduate pharmacology examinations in a medical college in Belagavi, Karnataka, India

Background: Multiple choice questions (MCQs) are a common method of assessment of medical students. The quality of MCQs is determined by three parameters such as difficulty index (DIF I), discrimination index (DI), and Distractor efficiency (DE). Item analysis is a valuable yet relatively simple procedure, performed after the examination that provides information regarding the reliability and validity of a test item. The objective of this study was to perform an item analysis of MCQs for testing their validity parameters.Methods: 50 items consisting of 150 distractors were selected from the formative exams. A correct response to an item was awarded one mark with no negative marking for incorrect response. Each item was analysed for three parameters such as DIF I, DI, and DE.Results: A total of 50 items consisting of 150 Distractor s were analysed. DIF I of 31 (62%) items were in the acceptable range (DIF I= 30-70%) and 30 had ‘good to excellent’ (DI >0.25). 10 (20%) items were too easy and 9 (18%) items were too difficult (DIF I <30%). There were 4 items with 6 non-functional Distractor s (NFDs), while the rest 46 items did not have any NFDs.Conclusions: Item analysis is a valuable tool as it helps us to retain the valuable MCQs and discard or modify the items which are not useful. It also helps in increasing our skills in test construction and identifies the specific areas of course content which need greater emphasis or clarity.

Item analysis to identify quality multiple choice questions/items in an assessment in Pharmacology of II MBBS students in Guntur Medical College of Andhra Pradesh, India

Background: In medical education, multiple choice questions/Items are the more frequently used assessment tools to assess the knowledge abilities and skills of medical students, for being their objectivity, wide coverage in less time. However only the Quality Items gives a valid and reliable assessment. The quality of an Item is determined by difficulty index (DIF I), Discrimination Index (DI) and Distractor efficiency (DE). Aim of the study was to know the quality of Items in pharmacology by Item analysis and to develop a MCQs bank with quality Items.Methods: The present study was conducted on 150 II MBBS students of Guntur Medical College, AP, India. A class test containing 50 Items with 150 distractors from topic chemotherapy was conducted. Item with the correct choice/response was awarded with one mark and with the wrong choice zero marks, no negative marks. Each test Item was analysed with DIF I, DI and DE and the results were tabulated and tested statistically, with unpaired "t" test.Results: Mean DIF I, DI, DE values with standard deviations in the present study are 44.72+17.63%, 0.30+0.12%, 84.48+24.65 respectively. DIF I of 32 (64%) items was good to excellent range (31%-60%) 9 (18%) Items were easy (>61%) and 9(18%) Items were difficult (>30%). DI of 10 (20%) Items was good (0.15 to 0.24.) 29 (58%) Items were excellent with DI > 0.25 and 11 (22%) Items were poor with DI <0.15. Among 150 distractors, 127 (85%) were functional distractors (FDs) and 23 (15%) were non-functional distractors (NFDs). DE of 33 (66%) items with nil NFDs was 100%, for 12 (24%) Items with one NFD, was 66.6%, for 4 (8%) items with 2 NFDs was 33.3% and for 1 (2%) Item with 3NFDs DE was 0%. When unpaired "t" test was applied to the means of "difficult" and "easy" Items, 96.22+11.33% SD, 51.44+29.31% SD respectively, the p-value obtained was 0.00058, which was highly significant.Conclusions: The study showed that Item analysis is a valid tool to identify quality Items, which assess, the students� knowledge abilities and discriminate different levels of performance abilities of students effectively.

Multiple Choice Questions - Role In Assessment Of Competency Of Knowledge In Anatomy

Background: The accurate, reliable and timely assessment of students is an essential domain of teaching duringMedical professional courses. The Multiple Choice Questions (MCQ) are time tested method of ready assessmentof undergraduate students. Although it evaluates student’s cognitive knowledge but does not evaluate professionalskills. However it is said that MCQs emphasize recall of factual information rather than conceptual understandingand interpretation of concepts.Objectives: The main objective of the study is to analyse the items with the help of item analysis and select theitems which are good for incorporation into future question bank with reliability.Materials and Methods: This study was done in Department of Anatomy, AIIMS, Patna. A 396 first year MBBSstudents of different batches took the MCQ test comprising 60 questions in two sessions. During the evaluationprocess of MCQ’s each correct response was awarded one mark and no marks was awarded for any incorrectresponse. Each item was analysed for difficulty index, discrimination index and distractor effectiveness.Results: The overall mean of Facilitative value, Discrimination Index, Distractor Effectiveness and CorrelationCoefficient was 66.09 (±21.55), 0.26 (±0.16), 18.84 (±10.45) and 0.55±0.22 respectively.Conclusion: The framing of MCQ should be according to Bloom’s classification to assess cognitive, affective aswell as psychomotor domain of the students. The MCQ having poor and negative discrimination should bereframed and again should be analysed.

A preliminary study on the relationship between the length of A2 type objective test questions and the degree of difficulty and discriminative power in medical examination / 中华医学教育探索杂志

Objective In a large-scale clinical medical examination, A2 type best multiple-choice test questions to the same knowledge were used respectively in simple and complex form, to compare the difficulty and discrimination indices of the two forms of test questions and provide evidence to the improve-ment of clinical medical examination. Method In a large-scale clinical medical examination more than 4000 candidates participated, and 20 questions to different knowledge points were randomly selected and used in the examination respectively in simple and complex A2 type best multiple-choice test questions. The difficulty and discrimination indices of the two forms of test questions were compared. Results The average difficulty coefficient of the 20 simple test questions (65.5 words per question in average) is 0.6829, and the average discriminative powers are 0.2675 and 0.2579 respectively using identification index method and point biserial correlation method. The average difficulty coefficient of the 20 complex test ques-tions (135.5 words per question in average) is 0.7095, and the average discriminative powers are 0.3065 and 0.2967 respectively using identification index method and point biserial correlation method. Conclusion To the same knowledge points, the average difficulty of complex A2 type best multiple-choice test questions is slightly lower than the simple ones, while the average discriminative power is increased in the complex questions. The complex A2 type test questions are of higher quality and more in line with the requirements of the medical entrance examination, medical education and examination reform.

Item Analysis, Reliability Statistics And Standard Error Of Measurement To Improve The Quality And Impact Of Multiple Choice Questions In Undergraduate Medical Education In Faculty Of Medicine At Unisza

Multiple-choice question as one best answer (OBA) is considered as a more effective tool to test higher order thinking for its reliability and validity compared to objective test (multiple true and false) items. However, to determine quality of OBA questions it needs item analysis for difficulty index (PI) and discrimination index (DI) as well as distractor efficiency (DE) with functional distractor (FD) and non-functional distractor (NFD). However, any flaw in item structuring should not be allowed to affect students’ performance due to the error of measurement. Standard error of measurement (SEM) to calculate a band of score can be utilized to reduce the impact of error in assessment. Present study evaluates the quality of 30 items OBA administered in professional II examination to apply the corrective measures and produce quality items for the question bank. The mean (SD) of 30 items OBA = 61.11 (7.495) and the reliability (internal consistency) as Cronbach’s alpha = 0.447. Out of 30 OBA items 11(36.66%) with PI = 0.31-0.60 and 12 items (40.00%) with DI = ≥0.19 were placed in category to retain item in question bank, 6 items (20.00%) in category to revise items with DI ≤0.19 and remaining 12 items (40.00%) in category to discard items for either with a poor or with negative DI. Out of a total 120 distractors, the non-functional distractors (NFD) were 63 (52.5%) and functional distracters were 57 (47.5%). 28 items (93.33%) were found to contain 1- 4 NFD and only 2 (6.66%) items were without any NFD. Distracter efficiency (DE) result of 28 items with NDF and only 2 items without NDF showed 7 items each with 1 NFD (75% DE) and 4 NFD (0% DE), 10 items with 2 NFD (50% DE) and 4 items with 3 NFD (25% DE). Standard error of measurement (SEM) calculated for OBA has been ± 5.51 and considering the borderline cut-off point set at ≥45%, a band score within 1 SD (68%) is generated for OBA. The high frequency of difficult or easy items and moderate to poor discrimination suggest the need of items corrective measure. Increased number of NFD and low DE in this study indicates difficulty of teaching faculty in developing plausible distractors for OBA question. Standard error of measurement (SEM) should be utilized to calculate a band of score to make logical decision on pass or fail of borderline students.

Relationship Between Difficulty Index and Distracter Effectiveness in Best-Answer Stem Type Multiple Choice Question.

Background: Single best-answer multiple-choice questions (MCQs) consist of a question (the stem) two or more choices from which examinees must choose the correct option (the distracters) and one correct or best response (the key). Item analysis is the process of collecting, summarizing and using information from students’ responses to assess the quality of test items. Classical test theory for item analysis is most followed method to determine the reliability by calculating Difficulty Index (P score) and Discriminating Index (D score) and Distracter effectiveness Aim: This Study was aimed to calculate P scoreand distracter effectiveness; to find out relationship between P score and distracter effectiveness. Material and methods: In this Cross Sectional study 65 items responded by 120 Students of first year M.B.B.S were studied for Item Analysis. Difficulty Index, and Distracter Effectiveness were calculated for each item. Distracters were identified and classified as Functioning and Non- functioning distracter. Interrelationship between P Score, and Distracter Effectiveness was calculated and analyzed by Epinifo 7 software Result: We found Items with two functioning distracters were more difficult than that of others followed by items with three functioning distracters. Conclusion: Distractors affect the item difficulty index and by the means also affects quality of the assessment .

Correlation between difficulty & discrimination indices of MCQs in formative exam in Physiology.

Introduction: Difficulty index (P) and discrimination index (D) are the parameters used to evaluate the standard of multiple choice questions (MCQs) in examination. Accordingly the standard of MCQs can be interpreted as excellent, acceptable or poor. This study was intended to find out the standard of MCQs in formative examination in Physiology. The study also intended to find out correlation between P and D. Materials and Methods: There were 240 MCQ items, taken from the past 4 year batches of 100 students and were analyzed for level of difficulty and discrimination index. The relationship between them for each test item was determined by Pearson correlation analysis using SPSS 11.5. Results: There was a wide distribution of item difficulty indices (8.57 to 95.71) and discrimination indices (-0.54 to 0.8).The mean difficulty index (P) was 52.53 + 20.59 and mean discrimination index was 0.30+ 0.18. On average, about 23% of the MCQ items were easy (P >70%), while about 15% were difficult (P <30%). The remaining 62% items were within acceptable range (30 to 70%). In all 4% of the items showed negative discrimination and 21% of the items exhibited poor discrimination. The remaining 75% of the items were in the range of acceptable to excellent discrimination. The discrimination index exhibited slight positive correlation with difficulty index (r = 0.191, P=0.003<0.01). The maximal discrimination (D=0.6-0.8) was observed with moderately easy/difficult items (P = 40% - 60%). Conclusion: The majority (75%) of the items was acceptable as far as difficulty and discriminative indices were concerned. Moderately easy/difficult items had maximal discriminative ability. Too easy and too difficult items gave poor discrimination index. Negative discrimination was observed in only 4% of the items indicating faulty items or incorrect keys.

Validación de la calidad de las preguntas en un examen final de la asignatura Morfofisiología Humana / Validation of the quality of questions in a final examination of the subject Human Morphophysiology

Introducción: el índice de dificultad y el poder de discriminación son indicadores fáciles de calcular y útiles para el análisis de la correspondencia entre los resultados esperados y los obtenidos de un instrumento evaluativo. Objetivo: evaluar la calidad de las preguntas del examen final ordinario de Morfofisiología Humana V. Métodos: fueron incluidos en esta investigación los 265 exámenes teóricos realizados por los estudiantes del segundo año en la Facultad de Medicina No. 2 de la Universidad de Ciencias Médicas de Santiago de Cuba, durante el curso 2011-2012, a los que se les calculó el índice de dificultad y el poder de discriminación en cada una de las 7 preguntas aplicadas. Resultados: las preguntas de respuesta alternativa, que evaluaban los contenidos sobre la sangre y el corazón, mostraron un índice de dificultad por debajo de 0,1 y un poder de discriminación inferior a 0,2, lo que hace necesaria su reformulación en próximos instrumentos evaluativos. Los valores más altos para ambos indicadores fueron 0,34 y 0,86, respectivamente, y correspondieron a una pregunta de selección múltiple sobre vasos sanguíneos y linfáticos, siguiéndoles en orden las preguntas de respuesta abierta. Conclusiones: se demostró la pertinencia de la mayoría de las preguntas, destacándose la capacidad de 5 de ellas para distinguir estudiantes de alto y bajo rendimientos.

Introduction: the difficulty index and the discrimination power are easy indicators to calculate and useful for the analysis of the correspondence between the expected and obtained results of an evaluative instrument. Objective: to evaluate the quality of questions of the regular final examination of Human Mophophysiology V. Methods: the 265 theoretical examinations carried out by the second year students in the Medical Faculty No. 2 of the Medical University in Santiago de Cuba, during the course 2011-2012 were included in this investigation, to which the difficulty index and the discriminating power were calculated in each of the 7 applied questions. Results: the questions with alternative answer which evaluated the contents on blood and heart, showed a difficulty index below 0.1 and a discriminating power lower than 0.2, making necessary their reformulation in future evaluative instruments. The highest values for both indicators were 0.34 and 0.86, respectively, and they corresponded to a question of multiple selections on blood and lymphatic vessels, followed in order by the questions of open answer. Conclusions: the relevancy of most of the questions was demonstrated, and the capacity of 5 of them to distinguish students of high and low performances was emphasized.

Nivel de dificultad y poder de discriminación del examen diagnóstico de la asignatura Morfofisiología Humana I / Level of difficulties and discriminatory power in the diagnostic test of Human Morphophysiology I subject

El objetivo de este trabajo es identificar el nivel de dificultad y poder de discriminación del examen diagnóstico de la asignatura Morfofisiología Humana I, del Nuevo Programa de Formación de Médicos en la Facultad-Policlínico Félix Edén Aguada, de Cienfuegos, Cuba. Para ello se calcularon los índices de dificultad y de discriminación, así como el coeficiente de discriminación a la totalidad de los 421 exámenes diagnósticos aplicados de la asignatura de Morfofisiología Humana I. Se entrevistaron profesores como informantes clave y estudiantes destacados. Todas las preguntas dieron la posibilidad de discriminar y predecir. De ellas, dos presentaron un índice de dificultad media, otras dos fueron medianamente fáciles y solo una fue altamente fácil. Se identificó el nivel dificultad y el poder de discriminación de cada pregunta del examen diagnóstico de Morfofisiología Humana I, lo que permitió plantear que el instrumento aplicado, fundamentalmente osciló entre dificultad media y medianamente fácil, con adecuado poder de discriminación. Los profesores y estudiantes reconocen la necesidad de estos exámenes para corregir dificultades en la evaluación, mejorar el trabajo metodológico y reajustar la autopreparación. Se hacen conclusiones.

The objective of this paper is to identify the level of difficulties and the discriminatory power of the diagnostic test of Human Morphophysiological I included in the New Medical Formation Program that is implemented in Felix Edén Aguada teaching polyclinics located in Cienfuegos province, Cuba. To this end, the difficulty and discrimination indexes were estimated together with the discriminatory coefficient of the 421 diagnostic tests of the above-mentioned subject. The professors as key informants and outstanding students were interviewed. All the questions made it possible to discriminate and to predict. Of these, two showed a mean difficulty index, other two were fairly easy and just one was very easy. The level of difficulties and the discriminatory power of each question of the exam were established, which allowed stating that the applied instrument ranged from mean difficulty to moderately easy, with suitable discriminatory power. The professors and the students agreed that these exams were necessary to overcome difficulties in the evaluation process, to improve the methodological work and to re-adjust self-preparation. The conclusions were also submitted.

Usability of Extended-matching Type Items in the Korean Medical Licensing Examinations (2002, 2003) / 한국의학교육

PURPOSE: In 2002, extended-matching type (R-type) items were introduced to the Korean Medical Licensing Examination. To evaluate the usability of R-type items, the results of the Korean Medical Licensing Examination in 2002 and 2003 were analyzed based on item types and knowledge levels. METHODS: Item parameters, such as difficulty and discrimination indexes, were calculated using the classical test theory. The item parameters were compared across three item types and three knowledge levels. RESULTS: The values of R-type item parameters were higher than those of A- or K-type items. There was no significant difference in item parameters according to knowledge level, including recall, interpretation, and problem solving. The reliability of R-type items exceeded 0.99. With the R-type, an increasing number in correct answers was associated with a decreasing difficulty index. CONCLUSION: The introduction of R-type items is favorable from the perspective of item parameters. However, an increase in the number of correct answers in pick 'n'-type questions results in the items being more difficult to solve.